Multi-focal structured illumination microscopy systems and methods

ABSTRACT

Various embodiments for a multi-focal selective illumination microscopy (SIM) system for generating multi-focal patterns of a sample are disclosed. The multi-focal SIM system performs a focusing, scaling and summing operation on each generated multi-focal pattern in a sequence of multi-focal patterns that completely scan the sample to produce a high resolution composite image.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a divisional application that claims benefit to U.S.non-provisional application Ser. No. 15/614,766 filed on Jun. 6, 2017,which is a continuation-in-part application that claims benefit of U.S.non-provisional application Ser. No. 14/380,392 filed on Aug. 22, 2014,which is a 371 of PCT patent application serial number PCT/US2013/027413filed on Feb. 22, 2013, which claims benefit to U.S. provisionalapplication Ser. No. 61/602,139 filed on Feb. 23, 2012, which are hereinincorporated by reference in their entirety.

FIELD

This document relates to multi-focal structured illumination microscopy,and in particular, to multi-focal structured illumination microscopysystems and methods for producing a plurality of multi-focal fluorescentemissions resulting from multi-focal patterns of a sample.

BACKGROUND

Classical fluorescence microscopy is limited in resolution by thewavelength of light, referred to as the “diffraction limit”, whichrestricts lateral resolution to about 200 nm and axial resolution toabout 500 nm at typical excitation and emission wavelengths when asample emits fluorescence that is detected by the microscope. Confocalmicroscopy is an optical imaging technique used to increase opticalresolution beyond the diffraction limit by using point illumination anda spatial pinhole arrangement to eliminate out-of-focus emission lightfrom specimens that are thicker than that of the focal plane, therebydelivering images with 1.41 times the resolution than the diffractionlimit by a method that requires tightly closing the pinhole.Unfortunately, closing the pinhole diminishes the signal level of theemitted light from the sample to such an extent as to make thisparticular method of super-resolution impractical. In addition, aconfocal microscope must perfectly align the excitation from themicroscope's illumination beam with the pinhole/detector, since amisaligned pinhole results in a reduced and weak light signal beingdetected as well as resulting in reduced axial optical sectioning of thesample itself. As such, misalignment of the confocal microscope cancause a reduction in the light signal.

A method for resolution enhancement for confocal microscopy has beenfound that uses an array of detectors, such as pixels in a camera image,wherein each of the detectors in the array produces a separate confocalimage. If the array of detectors is sufficiently small, each of theformed confocal images can be equivalent to similar confocal imagesformed by a confocal microscope with a tightly closed pinhole such that1.41 times the resolution of the diffraction-limited microscope isachieved when the confocal images are properly aligned. In addition,deconvolution provides a further increase in image resolution. However,this detector array arrangement is limited since only a singleexcitation point is scanned throughout a two-dimensional plane of thesample, which limits the speed the sample can be scanned and subsequentdetection of the fluorescence emissions of the sample.

Another type of microscopy, referred to as structured illuminationmicroscopy (SIM), illuminates a sample with spatially modulatedexcitation intensity, which is translated and rotated in differentpositions relative to the sample, with a wide-field image being taken ateach translation and rotation. Processing the raw images appropriatelyresults in a final image having double the lateral resolution ofconventional wide-field microscopy. Although such SIM systems generateimages with 2× the spatial resolution of a conventional microscope,there is still a sacrifice in temporal resolution when producing thefinal image, as time is required to acquire each of the multiple rawimages. SIM may also be used to reject out-of-focus blur, known as“optical sectioning”. However, such optical-sectioning is performedcomputationally, and is thus subject to shot (Poisson) noise. SIM isthus inappropriate for thick or highly stained samples, when backgroundfluorescence may cause this shot noise contribution to overwhelm thein-focus signal.

As such, there is a need in the art for a structured illuminationmicroscopy system that produces a multi-focal excitation pattern of thesample for each high resolution image without sacrificing scanningspeed, and that is resistant to the shot noise that may corrupt SIMimages.

SUMMARY

In an embodiment, a microscopy system may include a light source fortransmitting a single light beam and a beam splitter for splitting thesingle light beam into a plurality of light beams forming a multi-focalpattern. A scanner scans the plurality of light beams that forms themulti-focal pattern onto a sample such that the sample generates aplurality of fluorescent emissions resulting from each multi-focalpattern. A focusing component then defines an aperture configured tophysically block out-of-focus fluorescence emissions of the plurality offluorescent emissions resulting from each multi-focal pattern and allowsthrough in-focus fluorescent emissions to a pass through the aperture.In addition, a scaling component scales down the plurality of in-focusfluorescent emissions resulting from each multi-focal pattern such thateach of the plurality of in-focus fluorescent emissions is scaled downby a predetermined factor to produce a plurality of scaled in-focusfluorescent emissions resulting from each multi-focal pattern. A summingcomponent sums each of the plurality of scaled in-focus fluorescentemissions to produce a plurality of summed, scaled in-focus fluorescentemissions that form a composite image of the plurality of summed, scaledin-focus fluorescent emissions.

In another embodiment, a microscopy system may include a light sourcefor transmitting a single light beam; a beam splitter for splitting thesingle light beam into a plurality of light beams forming a plurality ofmulti-focal patterns, wherein each of the plurality of multi-focalpatterns defines a plurality of focal points; a scanner for scanning theplurality of light beams that forms each of the plurality of multi-focalpatterns onto a sample such that the sample generates a plurality offluorescent emissions resulting from each of the multi-focal patterns,wherein each of the plurality multi-focal patterns defines a pluralityof fluorescent focal points; a detector for collecting the plurality offluorescent emissions resulting from each of the multi-focal patterns;and a processing system for processing the collected multi-focalfluorescent emissions from the detector comprising: a processor inoperative communication with a database for storing the plurality ofcollected multi-focal fluorescent emissions, wherein the processorremoves out-of-focus fluorescent emissions resulting from each of theplurality of multi-focal patterns to leave only in-focus fluorescentemissions resulting from each of the plurality of multi-focal patterns,wherein the processor then scales the in-focus fluorescent emissionsresulting from each of the plurality of multi-focal patterns in a localcontraction operation in which each of the plurality of fluorescentemissions resulting from each of the multi-focal patterns maintains thesame proportional distance from another plurality of fluorescentemissions resulting from the multi-focal pattern as the plurality offluorescent emissions contract to produce scaled, in-focus fluorescentemissions; wherein the processor sums the plurality of multi-focalin-focus fluorescent emissions to produce a composite image.

In yet another embodiment, a method for multi-focal structuredillumination microscopy may include:

-   -   generating a single light beam;    -   splitting the single light beam into a plurality of light beams        in which the focal point of each the plurality of light beam        forms a plurality of multi-focal patterns,    -   illuminating a sample with the plurality of light beams forming        each of the plurality of multi-focal patterns, wherein the        illuminated sample produces a plurality of fluorescent emissions        resulting from each the plurality of multi-focal patterns;    -   performing a pinholing operation in which out-of-focus        fluorescent emissions from the plurality of fluorescent        emissions are blocked and only in-focus fluorescent emissions        from the plurality of fluorescent emissions are permitted to        pass through during the pinholing operation;    -   scaling each focal point of the plurality of in-focus        fluorescent emissions resulting from the plurality of        multi-focal patterns by a predetermined factor to produce a        plurality of scaled, in-focus fluorescent emissions resulting        from the plurality of multi-focal patterns; and    -   summing each of the plurality of scaled, in-focus fluorescent        emissions to form a composite image.

In yet another embodiment, a microscopy system may include a lightsource for transmitting a single light beam. A first microlens arraythat splits the single light beam into a plurality of light beams forforming at least one multi-focal pattern and a scanner that scans theplurality of light beams that forms the at least one multi-focal patternonto a sample such that the sample generates a plurality of fluorescentemissions with each of the at least one multi-focal pattern. Inaddition, a pinhole array blocks out-of-focus fluorescent emissions foreach of the at least one multi-focal pattern and allows through in-focusflorescent emissions to pass through the pinhole array. A secondmicrolens array produces a non-inverted image of the plurality of lightbeams having a one half a magnification, wherein the scanner rescans thenon-inverted image of the plurality of light beams. Finally, a cameracaptures the scanned non-inverted image.

In a further embodiment, a microscopy system may include a light sourcefor transmitting a single light beam; a spinning disk with a microlensfor splitting the single light beam into a plurality of light beamsforming a multi-focal pattern; a spinning disk with a pinhole array forblocking out-of-focus light beams of the plurality of light beams foreach multi-focal pattern and allowing through in-focus light beams ofthe plurality of light beams to pass through the spinning disk with apinhole array, wherein the spinning disk with a microlens is rotated intandem with the spinning disk with a pinhole array for scanning theplurality of light beams across a sample and generating a plurality offluorescent emissions; and a camera for capturing an image of theplurality of fluorescent emissions in each multi-focal pattern generatedby the sample.

In another embodiment, a microscopy system may include a light sourcefor transmitting a single light beam; a first spinning disk withconverging microlens for rotation in a first direction and positionedalong an optic axis for splitting the single light beam into a pluralityof light beams forming a multi-focal pattern. A second spinning diskwith a pinhole array for rotation in the first direction and positionedalong the optic axis for blocking out-of-focus light beams of theplurality of light beams for each multi-focal pattern and allowingthrough in-focus light beams of the plurality of light beams to passthrough the spinning disk with a pinhole array and a third spinning diskwith a diverging microlens for rotation in the first direction andpositioned along the optic axis, wherein the first spinning disk withconverging microlens, the second spinning disk with a pinhole array, andthe third spinning disk with a diverging microlens rotate in syncrelative to each other.

Additional objectives, advantages and novel features will be set forthin the description which follows or will become apparent to thoseskilled in the art upon examination of the drawings and detaileddescription which follows.

BRIEF DESCRIPTION OF THE DRAWINGS

The application file contains at least one photograph executed in color.Copies of this patent application publication with color drawings willbe provided by the Office upon request and payment of the necessary fee.

FIG. 1 is a simplified block diagram illustrating one method forgenerating a multi-focal pattern in a multi-focal structuredillumination (MSIM) system;

FIG. 2 is a simplified block diagram illustrating another method forgenerating a multi-focal pattern in another embodiment of themulti-focal SIM system;

FIG. 3 is a simplified illustration showing the various components forone embodiment of the multi-focal SIM system of FIG. 1;

FIG. 4 is a simplified illustration showing the various components forone embodiment of the multi-focal SIM system of FIG. 2;

FIG. 5 is a simplified illustration showing a multi-focal pattern with aplurality of focal points; and

FIG. 6 is a simplified illustration showing a scaling operation thatscales in-focus fluorescent emissions resulting from a multi-focalpattern emitted by a sample;

FIG. 7 is a flow chart illustrating one method for conducting a, scalingand summing operation by a processing system in the multi-focal SIMsystem of FIG. 4;

FIG. 8 is a flow chart illustrating one method for obtaining latticevectors when executing the out-of-focus fluorescence rejection, scalingand summing operation of the processing system;

FIG. 9 is a flow chart illustrating one method for obtaining offsetvectors when executing the out-of-focus fluorescence rejection, scalingand summing operation of the processing system;

FIG. 10 is a flow chart illustrating one method for obtaining shiftvectors executing the out-of-focus fluorescence rejection, scaling andsumming operations of the processing system;

FIG. 11 is a flow chart illustrating one method for pinhole masking whenexecuting the out-of-focus fluorescence rejection, scaling and summingoperation of the processing system;

FIG. 12 is a flow chart illustrating one method for scaling and localcontraction when executing the out-of-focus fluorescence rejection,scaling and summing operation of the processing system;

FIG. 13 is a flow chart illustrating one method for summing whenexecuting the out-of-focus fluorescence rejection, scaling and summingoperation of the processing system;

FIG. 14 is a simplified illustration showing the illumination systemused during testing of one embodiment of the multi-focal SIM system;

FIGS. 15A-15C illustrate various images of a sample embedded influoromount and FIG. 15D is a graphical representation of the same;

FIGS. 16A and 16B illustrate various images of dual-labeledthree-dimensional samples embedded in fluoromount; and

FIGS. 17A-17D illustrate various images of a sample using themulti-focal illumination system after pinhole focusing, scaling, andthree-dimensional deconvolution;

FIG. 18 is a simplified illustration showing an embodiment for aswept-field hardware MSIM system;

FIG. 19 is an image of a super-resolution images of microtubules withswept-field MSIM system;

FIG. 20 is a simplified illustration showing an embodiment for aspinning disk confocal microscopy system; and

FIG. 21 is a simplified illustration showing an embodiment for aspinning disk MSIM system.

Corresponding reference characters indicate corresponding elements amongthe view of the drawings. The headings used in the figures should not beinterpreted to limit the scope of the claims.

DETAILED DESCRIPTION

In modern microscopy, structured illumination microscopy (SIM) may beused to examine single cells using spatially patterned light to excitesample fluorescence that is later detected and one or more imagesprocessed to produce a super-resolution image with 2× the resolution ofa conventional wide-field microscopy image. However, the SIM microscopesacrifices speed for higher resolution (taking multiple raw images foreach super-resolution image). Furthermore, optical-sectioning in a SIMsystem is performed computationally, and is thus prone to shot noiseinherent in fluorescent background. This limits the thickness of thesample that can be examined, thereby requiring other microscopytechniques be used when examining thicker samples. For example, aconfocal microscopy system physically rejects out-of-focus light using apinhole arrangement that allows light from only a particular focal pointfrom the emission light being emitted by the sample to be detected bythe system, thereby producing high contrast, optically-sectioned imagesof relatively thicker samples than can be achieved by a SIM microscope.The confocal microscope is also capable of providing enhanced resolutionrelative to conventional wide-field microscopy. However, this enhancedimage resolution by the confocal microscope is attained by stopping downthe pinhole arrangement, which results in a corresponding prohibitiveloss in the fluorescence emission signal being detected from the sample.A modified confocal microscope has been shown to improve imageresolution to the resolution level of a SIM microscope withoutsacrificing emission signal strength; however, the slow scanning speedattained by the microscope makes it impractical for research purposes.

As such, embodiments of the multi-focal SIM (MSIM) system as set forthherein include particular hardware components, properties andcharacteristics that address issues related to achieving high imageresolution at a high scanning speed and signal strength required byconventional microscopy imaging systems, and that provide betterperformance in thick samples than SIM systems that are currentlycommercially available. The MSIM system described herein includesvarious embodiments of hardware components that generate a multi-focalexcitation pattern for each image taken of the sample to produce a highresolution image at high scanning rates without significant signal lossrelative to conventional confocal microscopy and SIM systems. Inaddition, the MSIM system performs scaling, pinholing, and summing stepsusing just an arrangement of hardware components, such as pinholesmirrors, and micro-lens arrays to perform the operation for producingmultifocal excitation pattern using a structured illuminationmicroscopy, rather than using a processor and software arrangement toperform the same operation. Further details of the multi-focal SIMsystems and methods are discussed in greater detail below.

Referring to the drawings, various embodiments of the multi-focal SIM(MSIM) system are illustrated and generally indicated as 100, 200, 300,400, and 500 in FIGS. 1-21. As illustrated in FIG. 1, one generalembodiment of the MSIM system, designated 100, is shown. MSIM system 100performs an illumination pattern generation operation 102, which splitsa single light beam into a plurality of light beams for generating oneor more multi-focal patterns, in which each multi-focal pattern definesan arrangement of focal points for each of the plurality of light beams.For example, a simplified illustration of one example of a multi-focalpattern 170 is shown in FIG. 5. In one embodiment, the multi-focalpattern 170 may include a plurality of focal points 172 arranged in aparticular multi-focal pattern 170 defined by the plurality of lightbeams with each multi-focal pattern 170 defining a different arrangementof focal points for the plurality of light beams such that multi-focalpattern 170 maximizes the distance between any two nearest focal points172 for a given density of focal points 172, thereby minimizingcrosstalk. As used herein, the term crosstalk refers to the ability ofnearby light beams to cause excitation and fluorescence from other focalpoints to appear as if focal points originate from the focal point inquestion. In a scanning operation 104, the plurality of light beams ineach multi-focal illumination pattern are rastered onto a sample 106being illuminated such that the sample 106 emits a plurality offluorescent emissions. The fluorescent emissions emitted by the sample106 are rastered in a de-scanning operation 108 which redirects theplurality of fluorescent emissions for removal of out-of-focusfluorescent emissions in a focusing operation 110. In the focusingoperation 110, out-of-focus fluorescent emissions are blocked and onlyin-focus fluorescent emissions are allowed to pass through forprocessing.

The in-focus fluorescent emissions caused by each multi-focal patternare then scaled using a scaling operation 112 that locally contractseach of fluorescent emissions by a predetermined factor. In oneembodiment of the scaling operation 112 illustrated in FIG. 6, a localcontraction of the fluorescent foci 172 caused in a single multi-focalpattern 170 occurs. For example, the local contraction of fluorescentfoci 172A and 1726 to scaled fluorescent foci 172A′ and 1726′,respectively, in the multi-focal pattern 170 is such that the distance300 between the geometric centers 175 of fluorescent foci 172A and 1726and the distance 302 of fluorescent foci 172A and 1726 remains the sameregardless of the degree of scaling applied to the fluorescent foci 172of the multi-focal pattern 170. In other words, the scaling operation112 contracts the fluorescent foci 172 locally while keeping therelative distances between each of the fluorescent foci the same. Afterthe scaling operation 112, the scaled in-focus fluorescent emissions foreach multi-focal illumination pattern are then rastered in a rescanningoperation 114 that allows the contracted, in-focus fluorescent emissionscaused by each multi-focal pattern to be collected by a detector andsummed to produce a composite high resolution image in a collection andsumming operation 116.

In some embodiments, after the collection and summing operation 116 thecomposite image may undergo a deconvolution operation 118 that performsa level of de-blurring that further enhances the resolution of thecomposite image. The deconvolution operation 118 may be any conventionaldeconvolution operation 118, such as the commercially available PiotyrWendykier's Parallel Iterative Deconvolution Plugin.

Referring to FIG. 2, another method for generating multi-focal patternsaccording to another embodiment of the MSIM system, designated 200, isillustrated. MSIM system 200 performs an illumination pattern generationoperation 202, in which a single light beam is split into a plurality oflight beams for generating one or more multi-focal patterns of theplurality of light beams. In a scanning operation 204, the plurality oflight beams for each multi-focal pattern is rastered onto a sample 206.The sample 206 produces fluorescent emissions in response to themulti-focal patterns that are detected in a collection operation 208. Inthe collection operation 208 a detector collects the in-focusfluorescent emissions and transmits the collected data to a processingsystem that performs a processing operation 210. The processingoperation 210 removes out-of-focus fluorescent emissions in eachmulti-focal pattern, scales the remaining in-focus fluorescentemissions, and then sums the scaled, in-focus fluorescent emissions togenerate a composite image composed from the plurality of fluorescentemissions created by the multi-focal patterns. In some embodiments, thecollected data may then undergo a deconvolution operation 212 similar tothe deconvolution operation 120, which performs a de-blurring of thecomposite image to further enhance image resolution.

Referring to FIG. 3, one embodiment of the multi-focal SIM system 100may include an illumination source 120, for example a laser, forgenerating a single light beam 122 that is transmitted through a beamsplitter 124, such as a micro-lens array. The beam splitter 124 splitsthe single light beam 122 into a plurality of light beams 126 with eachof the plurality of light beams 126 having a different focal point thatcollectively form a multi-focal pattern 128. In some embodiments, afirst lens 130 images each multi-focal pattern 128 through a dichroicmirror 132 in direction A and onto a scanning apparatus 134, such as agalvanometer, to perform the scanning operation 104.

During the scanning operation 104, the scanning apparatus 134 rasterseach multi-focal pattern 128 of the plurality of light beams 126 onto asample 144 through an arrangement of a second lens 136, a tube lens 138and an objective lens 140 and onto the sample 144.

In response to the sample 144 being illuminated by the multi-focalpatterns 128 of the plurality of light beams 126, the sample 144 emitsfluorescent emissions 142 caused by the multi-focal patterns 128composed of light beams 126. The plurality of fluorescent emissions 142for each multi-focal pattern 128 emitted by the illuminated sample 144is then captured through the objective lens 140, and passed through thetube lens 138 and second lens 136 and onto the scanning apparatus 134,which de-scans each of the plurality of fluorescent emissions 142 byrastering the fluorescent emissions 142 from the second lens 136 andonto the dichroic mirror 132 in a direction B opposite that of directionA in which the plurality of light beams 126 pass directly through thedichroic mirror 132. In direction B, the plurality of fluorescentemissions 142 are redirected by the dichroic mirror 132 to pass througha third lens 146 in which the plurality of fluorescent emissions 142 arethen focused onto a pinhole array 148 to perform the pinholing operation110.

During the pinholing operation 110, the pinhole array 148 physicallyblocks and rejects out-of-focus fluorescent emissions 142 and allowsonly in-focus fluorescent emissions 142A to pass through the pinholearray 148. In one embodiment, the pinhole array 148 may include aplurality of apertures configured to permit only in-focus fluorescentemissions 142A to pass through the pinhole array 148, while blocking anyfluorescent emissions 142 that do not pass through one of the apertures.

After passing through the pinhole array 148, the plurality of in-focusfluorescent emissions 142A are scaled using the scaling operation 112discussed above that locally contracts each of the focal points for arespective multi-focal pattern 128 by a predetermined factor, forexample a factor of two, using a first micro-lens array 150 arranged inseries with a second micro-lens array 152. In one embodiment, the firstmicro-lens array 150 collimates the multi-focal pattern 128 of in-focusfluorescent emissions 142A, while the second micro-lens array 152receives the collimated in-focus fluorescent emissions 142A and modifiesthe focal length for the collimated in-focus fluorescent emissions 142Asuch that each foci are scaled down by a predetermined factor to achievelocal contraction of the multi-focal pattern 128. In some embodiments,the scaled in-focus fluorescent emissions 142B caused by eachmulti-focal pattern 128 are then redirected by a first mirror 154 topass through a fourth lens 158 for focusing the scaled in-focusfluorescent emissions 142B onto a second mirror 156. The second mirror156 redirects the scaled in-focus fluorescent emissions 142B to passthrough an emission filter 160 that permits only scaled, in-focusfluorescent emissions 142B with a particular wavelength, for example 515nm, to pass through the emission filter 160. Once the scaled, in-focusfluorescent emissions 142B are filtered, the scanning apparatus 134rasters the fluorescent emissions 142B onto the detector 164 through afifth lens 162 such that the scaled, in-focus fluorescent emissions 142Bfor each multi-focal pattern 128 are collected and the in-focusfluorescent emissions 142B are summed by the detector 164 to produce acomposite image 143. The process of collecting the fluorescent emissions142B caused by multi-focal patterns 128 is repeated until the entirefield of view has been illuminated and all the resulting fluorescentemissions 142B are collected by the detector 164.

In some embodiments, the detector 164 may be a camera with the shutterleft open to exposure as the scaled, in-focus fluorescent emissions 142Bare rastered onto the detector 164 for collection until the entire fieldof view is illuminated and all the collected data is received.

In some embodiments, the composite image 143 is transmitted to aprocessing system 166 that performs the deconvolution operation 118 thatperforms a de-blurring function to enhance the resolution of eachcomposite image 143.

Referring to FIG. 4, one embodiment of the multi-focal SIM system 200for generating a multi-focal pattern 223 may include an illuminationsource 201 that generates a single light beam 203 that is transmittedthrough a beam expander 205 that produces an expanded single light beam203, which is reflected off a beam steering mirror 207 for performingthe scanning operation 204 that scans the expanded light beam 203 onto abeam splitter 209, for example a commercially available digitalmicromirror device (DMD) or a swept field confocal unit. In someembodiments, the DMD generates and switches multi-focal patterns 223with each focal point being an illuminated spot on the multi-focalpattern 223. Each illumination spot is created by a single DMD mirrorpixel being in the ON position such that a portion of the expanded lightbeam 203 is reflected off the single DMD mirror pixel. The beam splitter209 performs the illumination pattern generation operation 202 thatsplits the expanded light beam 203 being scanned into a plurality ofexpanded light beams 203A that collectively form a sequence ofmulti-focal patterns 223. The plurality of expanded light beams 203A foreach multi-focal pattern 223 is passed through a first tube lens 211 topass directly through a dichroic mirror 213 along a direction A.

After passing directly through the dichroic mirror 213 the expandedlight beams 203A are focused by an objective lens 215 onto a sampleplane for performing sample illumination 206 of sample 217 and generatea plurality of fluorescence emissions 205 emitted by the sample 217. Inthe collection operation 208, the fluorescent emissions 222 are focusedback through the objective lens 215 and onto the dichroic mirror 213along direction D opposite that of direction C such that the fluorescentemissions 222 for each multi-focal pattern 223 are redirected and passthrough a second tube lens 219. The second tube lens 219 focuses thefluorescent emissions 222 onto a detector 221, thus collecting eachmulti-focal pattern 223 in the form of collected data 227 fortransmission to a processing system 225 that rejects out-of-focus lightand performs scaling and summing operation 210. The process ofcollecting the fluorescent emissions 222 caused by each multi-focalpattern is repeated until the entire field of view has been illuminated.In some embodiments, the processing system 225 may include a processor230 in operative communication with the detector 221 for processingcollected data 227 stored in a database 232, such as a computer-readablemedium, by executing instructions a shall be discussed in greater detailbelow.

In one embodiment, the processing system 225 rejects out-of-focusfluorescent emissions 222, and performs the scaling and summingoperation 210 on the collected data 227 for each multi-focal pattern 223using a computerized procedure for processing the collected data 227.Referring to FIG. 7, a flow chart illustrates one method for performingthe out-of-focus fluorescence rejection, scaling and summing operation210 using the processing system 225. At block 1000, the processingsystem 225 performs an automatic lattice detection of the multi-focalillumination pattern 223. Specifically, automatic lattice detectiondetermines the position of each illumination focal point for aparticular multi-focal pattern 223 in order to perform pinhole maskingand local contraction of the various fluorescent foci. For example, theautomatic lattice detection determines five vectors to completelyspecify the location of all focal points for each fluorescent emission222 in a particular multi-focal pattern 223 in an image acquisitionseries. In some embodiments, the five vectors may be lattice vectors,shift vectors and an offset vector. Two lattice vectors specify thetwo-dimensional displacement between any two neighboring illuminationspots (e.g., focal points). As such, any lattice point displaced by alattice vector will fall on another lattice point. Due to thermal shift,the exact locations of each illumination spot may vary in time, soaccurately extracting all vectors for each measured dataset from thecollected data 227 provides the best results using the followingmethodology.

Referring to FIG. 8, a method for accurately extracting all vectors foreach measured dataset of collected data 227 is shown when the processingsystem 225 performs the out-of-focus rejection, scaling and summingoperation 210. The offset vector specifies the absolute position of theillumination spot closest to the center of a multi-focal pattern in anyone image acquisition series being processed by the processing system225. The shift vectors specify the distance of each illumination spotfor a particular multi-focal pattern 223 moves between consecutiveimages. Since the multi-focal pattern 223 is rastered in atwo-dimensional plane, a “fast” shift vector is applied at every step ofthe operation 210 and a “slow” shift vector is applied when a “fast”shift vector has completed illuminating one row. As used herein, theterm “fast” shift vector specifies those lattice points in a single rowand the term “slow” shift vector specifies successive rows. At block2000, each raw image representing the fluorescence 142B captured from aparticular multi-focal pattern 223 is multiplied by a Hann window (whichprevents ringing in the Fourier domain) and Fourier transformed. Atblock 2002, the resulting Fourier magnitudes are averaged to obtain ahigh signal-to-noise measurement of peak locations in the Fourier space.At block 2004, the averaged resulting Fourier magnitudes are filteredthrough a bandpass filter. At block 2006, a search for spikes or peaksin the Fourier domain is accomplished. Since the raw images for summedFourier magnitudes are expected to be a periodic lattice of peaks, thedimensions of each periodic lattice is made by determining the localFourier maxima, which are either lattice elements or noise. At block2008, the harmonics for the three lowest spatial frequency peaks aredetermined by computationally searching for peak intensities at and nearthe expected frequency multiples. At block 2010, given the locations ofthe three lowest spatial frequency peaks with harmonics, a search isconducted for peaks at their vector sums and differences, which composethe lattice vectors. In the present method, the peaks in a periodiclattice predict the location of other peaks in the same lattice withevery lattice point being located at the vector sum of an integer numberof Fourier lattice vectors. At block 2012, the processing system 225solves the resulting system of equations by linear squares to obtainthree lattice vectors. Any single peak in the Fourier domain will havesome positional error due to noise or pixelization. Utilizing the methodof least squares, the information provided by each peak produces a moreaccurate measurement of the lattice dimensions. At block 2014, theprocessing system 225 sums the lattice vectors to determine an errorvector and then subtracts one-third of the error vector from eachFourier lattice vector. At block 2016, any two of the two Fourierlattice vectors are selected and transformed to obtain real spacelattice vectors. It has been discovered that obtaining two latticevectors for the multi-focal illumination pattern makes it substantiallyeasier to measure the offset vector and the shift vector.

Referring to FIG. 9, a method for determining offset vectors is shownwhen the processing system 225 performs the rejection of out-of-focusblur, scaling and summing operation 210. At block 3000, a first rawimage is median filtered. At block 3002, a lattice of points from themeasured lattice vectors is constructed with an offset vector of zero.At block 3004, a set of smaller images centered at each lattice point isextracted from the median-filtered raw image using interpolation forsub-pixel centering. At block 3006, the smaller images are averaged toform a “lattice average” image. At block 3008, the brightest pixel inthe lattice average image is determined. At block 3010, the sub-pixelposition of each intensity peak in the lattice average image isestimated using interpolation techniques.

Referring to FIG. 10, a method for determining the shift vectors isshown when the processing system 225 performs the rejection ofout-of-focus blur, scaling and summing operation 210. At block 4000, aset of expected Fourier peak locations is constructed from the measuredFourier lattice vectors. At block 4002, extract phase is extracted foreach image number for the transformed collected data 227. At block 4004,the processing system 525 phase-unwraps each series of multi-focalpatterns 223 to remove 2π jumps. It has been noted that the magnitude ofthe Fourier transform of each raw image is independent of shifts of theraw images. Information about shifting is encoded in the phase of theFourier transforms. At block 4006, the processing system 525 reshapeseach phase series in two-dimensional series to a two-dimensional arraywith the same dimensions as the scan pattern (e.g., 16 pixels×14pixels). After reshaping of each phase series, at block 4008, a line isfit to the average slope of each phase series in both the “fast” and“slow” directions. In each consecutive raw image that is detected, theillumination shifts by X_(FAST). At the end of each row, theillumination also shifts by X_(SLOW). If a raw image shifts by X, thephase of a peak in the Fourier space located at k shifts by k*X. Atblock 4010, the processing system 525 solves the resulting system ofequations for X_(FAST) and X_(SLOW). At block 4012, an offset vector isconstructed for the first and last frames of the series of multi-focalpatterns 223. Computing the offset vector for the last frame, which usesthe same process that computed the offset vector of the first frame,provides a strong constraint that greatly increases the accuracy of theestimate. At block 4014, a predicted offset vector is constructed basedon the current estimate for X_(FAST) and X_(SLOW). Once the predictedoffset vector is constructed, the difference between the two vectors isdetermined to be an error vector, which is divided by the number of rawimages in the scan, and subtract the result from X_(SLOW).

Referring to FIG. 11, a method for pinhole masking is shown forrejecting out-of-focus blur when the processing system 225 performs theout-of-focus blur, scaling and summing operation 210. At block 5000, aset of expected illumination locations based on the vectors determinedabove is constructed for each raw image collected during the collectionoperation 208. At block 5002, smaller subimages centered at eachillumination location are extracted using interpolation techniques andthen at block 5004 each centered subimage is multiplied by theprocessing system 525 by a two-dimensional Gaussian mask. These processsteps simulate the effect of physical pinholes. At block 5006, thebackground may be optionally subtracted from each subimage and thecentered subimage multiplied by a correction factor given by calibrationdata. At block 5008, individual hot pixels may be optionally medianfiltered by the processing system 525.

Referring to FIG. 12, a method for scaling and performing localcontraction is shown when the processing system 525 performs therejection of out-of-focus blur, scaling and summing operation 210. Atblock 6000, each masked subimage is re-sampled to shrink the maskedsubimage by a factor of about two. At block 6002, each scaled, maskedsubimage is added to the processing raw image.

Referring to FIG. 13, a method for summing is shown when the processingsystem 525 performs the rejection of out-of-focus blur, scaling andsumming operation 210. At block 7000, the processed raw images aresummed to form a composite image. At block 7002, the raw images may beoptionally summed to produce a calculated “widefield” image.

The resulting composite image produced by the rejection of out-of-focusblur, scaling and summing operation 210 has been shown to have a √2better resolution than a typical widefield image produced by aconventional widefield microscopy. The rejection of out-of-focus blur,scaling and summing operation 210 also greatly improves opticalsectioning, similar to a conventional spinning-disk or swept-fieldconfocal microscope.

Referring to FIG. 18, an embodiment of the MSIM system, designated 300,is shown. MSIM system 300 may be a swept-field hardware arrangement inwhich a similar degree of resolution enhancement as discussed above forMSIM systems 100 and 200 is achieved. The major difference between MSIMsystem 300 and MSIM systems 100 and 200 is that the operations relatedto scaling, pinholing, and summing steps are achieved in MSIM system 300using hardware rather than software as required for MSIM systems 100 and200. In this embodiment, MSIM system 300 may include a light source 302that emits a light beam 305 which is transmitted through a converging(+) microlens array 304 and the resulting focused light beam relayedthrough a first scan lens 306, scanned by a galvanometer mirror 308, andthen through a second scan lens 310 in which the first and second scanlenses 306 and 310 are in a 4 f configuration to an intermediate imageplane.

As further shown, the galvanometric mirror 308 may be positioned at thefocal point between the first and second scan lenses 306 and 310 andsweeps the excitation foci across the sample plane 316, thus producing aswept-field excitation that covers the imaging field. In addition, atelescope arrangement of a tube lens 312 and an objective lens 314 ispositioned between second scan lens 310 and the sample plane 316 whichdemagnifies the intermediate stage and produces an array of excitationfoci which is swept across the sample plane 316 by the galvanometricmirror 308. The resulting fluorescence generated by the excitation focibeing swept across the sample plane 316 follows the same pathway backthrough the objective lens 314 and tube lens 312, which is descanned bythe galvanometer mirror 308, but diverted with a dichroic mirror 318positioned between the converging microlens array 304 and the first scanlens 306. Once diverted, the fluorescence emission is passed through apinhole array 320, thereby greatly reducing out-of-focus fluorescenceemission.

The resulting in-focus fluorescence emission is then relayed using a 4 ftelescope pair consisting of a first relay lens 322 which focuses thein-focus fluorescence emission onto a first mirror 324 that diverts thein-focus fluorescence emission through a second relay lens 330 which isthen diverted by a second mirror 326 and a third mirror 328 insuccession to a second converging (+) microlens array 332. In oneembodiment, the second converging (+) microlens array 332 may bepositioned one focal length before the focus that would have been formedby the second relay lens 330, thereby producing an erect (non-inverted)image of the fluorescence emission foci with one half the magnification.This erect image is then relayed through another telescope arrangementof a third scan lens 334 and fourth scan lens 336 arranged in a f4configuration in which the erect image may be rescanned by thegalvanometer mirror 308 positioned at the focal point between the thirdand fourth scan lens 334 and 336. A camera 338 having an emission filter340 captures the final image. The MSIM system 300 can produce enhancedresolution images such as shown in FIG. 19 which shows AlexaFluor 488 nmlabeled microtubules in a fixed U2OS cell. The apparent width of each ofthe microtubules is about 200 nm. The image shown in FIG. 19 is rawwithout any post-processing deconvulotion that would be expected tofurther increase the resolution of the image.

FIG. 20 shows an example of a spinning disk confocal microscopy system,designated 400. System 400 may include a light source 400 for generatinga light beams that are passed through a spinning disk with converging(+) microlens 404 and then a spinning disk with a matched pinhole array406. The resulting excitation is then imaged onto the sample 412 with anoptical arrangement of a tube lens 408 and objective lens 410. In thisarrangement, the spinning of both the spinning disk with converging (+)micolens 404 and the spinning disk with matched pinhole array 406 intandem, the excitation foci that is created cover the field of view ofthe sample 412. Fluorescence emissions originating from the sample 416are then passed back through the objective lens 410, the tube lens 408and the spinning disk with matched pinhole array 406. The fluorescenceemissions are then diverted with a dichroic mirror 414 such that thefluorescence emissions pass through a telescope arrangement 416 whichare captured by a camera 420 through an emission filter 418.

Referring to FIG. 21, another embodiment of the MSIM system with aspinning disk multi-focal structured illumination microscopy which basedon the modified spinning disk system 400 described in FIG. 20,designated 500, is shown. In one embodiment, a conventional spinningdisk confocal microscope is converted into a resolution doubling devicewith one major modification. MSIM system 500 may include a light source502 that generates a light beam that is diverted by a dichroic mirror504 which passes through a spinning disk with converging (+) microlens506 and then a spinning disk with a pinhole array 508 before beingfocused onto the sample 514 by a tube lens 510 and objective lens 512.Once the sample 514 is illuminated, the fluorescence emissions generatedby the sample 514 pass back through the objective lens 512 and tube lens510 which focus the fluorescence emissions through a spinning disk withdiverging (−) microlens 516 positioned along the optic axis which ismade to spin in sync with the spinning disk with converging (+)microlens 506 and the spinning disk with pinhole array 508 before thefluorescence emissions pass through a telescope arrangement 518 beforebeing captured by a camera 520 after being filtered by emission filter522. In one embodiment, the spinning disk with diverging (−) micolens516 should contain the same number of microlenses as the spinning diskwith converging (+) microlens 506 and that these microlenses should bespaced at the same spatial location on each spinning disk 506 and 516.In some embodiments, if the microlenses for the spinning disk withdiverging (−) microlens 516 have a half focal length of the spinningdisk with converging (+) microlens 506, and the spinning disks 506 and516 are spaced apart by the difference of the focal lengths, thisarrangement will form a Galilean telescope with a magnification of ½. Ifsuch a telescope is positioned at the appropriate distance from theerect, DE magnified images are so created, the rotation of the spinningdisks 516 and 506 in synchrony with the camera exposure performs thedesired pinholing, scaling, and summing operations required forresolution-doubling. One important advantages of the spinning diskarrangement of the MSIM system 500 is the significantly reduced numberof emission optics, nine instead of seventeen. Another advantage is thatif the spinning disk with diverging (−) microlenses 516 can be readilyfabricated, the optical set up for the MSIM system 300, 400 and 500 islikely much easier to align than a conventional swept-field microscope,making the translation of the MSIM technology to spinning disk readilyavailable.

Testing

To investigate the potential of the multi-focal SIM system 200 forbiological imaging, antibody-labeled microtubules in human osteosarcoma(U2OS) cells embedded in fluoromount were imaged. The following is adescription of the illumination system (FIG. 14), microscope system,sample preparation, and data processing of captured images. The use ofmulti-focal patterns in combination with deconvolution allowed us toinvestigate a variety of samples at imaging rates of 1 Hz, atresolutions down to 145 nm laterally and 400 nm axially. Compared toconventional structured illumination microscopy systems, the multi-focalSIM system 200 provided three-dimensional images of samples 5-8 timesthicker than conventional structured illumination microscopy systems. Inthe present investigation, microtubules were imaged in live transgeniczebrafish embryos at depths grater than 45 μm from the coverslipsurface. In addition, four-dimensional SIM datasets of GFP-labeledhistones in live nematode embryos were obtained.

For testing, a periodic lattice of approximately equilateral trianglesfor our illumination point locations because this particular patternmaximized the distance between any two nearest neighbors for a givendensity of points, thereby minimizing crosstalk. The multi-focalillumination pattern was translated one Digital Micromirror Device (DMD)pixel at a time, which corresponded to a step size of 120 nm in thesample plane. Larger steps did not evenly illuminate the sample, givinga visible striping artifact, while smaller steps increased acquisitiontime and dose with no increase in image quality.

Multi-focal patterns were imaged onto the sample, which was mounted on acommercial inverted microscope, and a scientific-grade complementarymetal-oxide-semiconductor camera (sCMOS) was used to record one rawimage for each multi-focal pattern position. By varying the spacingbetween the illumination points, acquisition speed may be traded forsectioning quality. It was discovered that widely spaced foci had lesscrosstalk, but additional multi-focal illumination patterns wererequired to evenly illuminate a sample. In contrast, denser foci hadmore crosstalk, but required correspondingly fewer multi-focal patternsto evenly illuminate the sample. It was found that a multi-focal patternwith a 16 pixel horizontal and a 14 pixel vertical separation betweenscan points provided good results in the biological samplesinvestigated. The resulting 224 raw exposures taken at 222 Hz for a 480pixel×480 pixel field of view corresponded to about a 1 Hzsuper-resolution image acquisition rate.

To investigate the potential of multi-focal SIM system 200 forbiological imaging, we imaged antibody-labeled microtubules in humanosteosarcoma (U2OS) cells embedded in fluoromount as shown in the imagesillustrated in FIGS. 15A-15D. Compared to widefield images, themulti-focal illuminated, pinholed, summed, and scaled images produced bythe SIM system 200 improved image resolution including image contrast.In addition, parallel, iterative deconvolution further improved theresulting composite image, which revealed features previously obscuredby diffraction. The apparent full-width at half maximum (FWHM) intensityof microtubules in multi-focal SIM system 200 images was 145 nm, whichwas a two-fold improvement compared to widefield imaging. Similarexperiments on 110 nm subdiffractive beads confirmed this result(multi-focal SIM system 200 FWHM 146+/−15 nm vs. widefield FWHM 284+/−32nm, N=80 beads. The total acquisition time for the 48 μm×49 μm field wasabout 1 s, a 6500-fold improvement over a conventional image scanningmicroscopy (ISM) system assuming the same 222 Hz raw frame rate for eachmicroscopy system.

The suitability of the multi-focal SIM system 200 for dual-labeled,three-dimensional samples was also investigated as shown in the samplesillustrated in FIGS. 16A-16B. A Z stack of images on a fixed cellembedded in fluoromount was used. Immuno-labeled microtubules with AlexaFluor 488 and stained mitochondria with Mitotracker Red obtained avolume of 3.7 μm thickness, with individual slices separated by 100 nm.Compared to widefield images obtained with the same total illuminationdose, three-dimensional multi-focal SIM system 200 images provided astriking increase in image contrast, due to the combined physical (viadigital pinholes) and computational (via three-dimensionaldeconvolution) removal of out-of-focus light.

The resulting composite images produced by the multi-focal SIM system200 had approximately a two-fold resolution improvement over widefieldimaging; better resolving microtubules and “worm-like” mitochondria. Forexample, better resolving of sub-diffractive voids at the ends ofindividual mitochondria was achieved including microtubule pairsseparated by greater than 200 nm. Unexpectedly, multi-focal SIM system200 also improved the axial resolution approximately two-times overwidefield images, as microtubules had apparent axial FWHM of about 400nm. This result was confirmed on 100 nm subdiffractive beads(Multi-focal SIM system FWHM 402+/−49 nm; widefield 826+/−83 nm, N=80beads.

The MSIM system 200 was also applied to three-dimensional imaging ofthicker live samples in which the pinhole operation physically rejectsout-of-focus light that would otherwise swamp the in-focus light signal.To demonstrate this capability, live, immobilized zebrafish embryosexpressing a GFP transgene that labeled microtubules were imaged.

Using multi-focal illumination in accordance with the multi-focal SIMsystem 200, 241 slices were acquired spaced 0.2 μm apart at atwo-dimensional imaging rate of 1 Hz. After pinhole focusing, scaling,and three-dimensional deconvolution, a volume of 48.2 μm thickness wasachieved as shown in the images of the sample illustrated in FIGS.17A-17D. Structural features such as the boundary between two adjacentsomites, alignment of microtubules along the somite boundary, andmicrotubule free-regions corresponding to the nuclei of the developingmuscle cells are clearly visible in the stack. Estimation of thez-position of successively deeper nuclei within this stack suggests thatthe imaging volume contained 6-7 cell layers.

In one test, the imaging rate of the multi-focal SIM system 200 captureda dividing cell in the epidermis without significant motion blur in theimages. The resolution enhancement of multi-focal SIM system 200 wasretained throughout the volume, as the separation between microtubulepairs at the site of the cell division was resolved to better than 200nm, and microtubules in the epidermis had lateral FWHM 175+/−33 nmlaterally (N=30) and 496+/−65 nm axially (N=21).

Illumination System

In the illumination system, all optics were mounted on an optical table(Kinetic Systems, Vibraplane Model #5704-3660-23SPL) to minimizemechanical vibrations. For exciting fluorescence, two lasers were used:a 150 mW, 561 nm laser (561, Coherent, Sapphire 561-150 CW CDRH) and a200 mW, 488 nm laser (488, Coherent, Sapphire 488-200 CDRH). Mechanicalshutters (Thorlabs, SH05 and SC10) placed after each laser was used tocontrol illumination. Beams were combined with a dichroic mirror (DC,Chroma, 525dcxru) and expanded 6.7 times with a beam expanderconstructed from two achromatic lenses (Edmund, f=30 mm, NT49-352-INKand Thorlabs, f=200 mm, AC254-200-A-MLO. Expanded beams were directedonto a digital micromirror device (DMD, Digital Light Innovations, D4100DLP 0.55″ XGA) 24 degrees off normal, so that in the ON position themicromirrors tilted the output beam normal to the DMD face. The centerorder of the resulting illumination pattern was demagnified 1.5 timeswith a beam de-expander (Thorlabs, f=75 mm, AC254-075-A-ML and f=50 mm,AC254-050-A-MLO, aligned in a 4 f configuration such that the DMD facewas re-imaged at the back focal plane of a 180 mm tube lens internal tothe microscope (Olympus, IX-81). These elements are shown in FIG. 14.After entering the left side port of the microscope, the beamsequentially passed through (i) the tube lens; (ii) a dichroic mirror(Chroma, zt405/488/561); (iii) a 60× objective (Olympus, PlanApo, NA1.45 TIRF, for single cells, or UPLSAPO 60XS, NA 1.3, for zebrafish andworm embryos) for a total demagnification of 90× between the DMD and thesample being illuminated. The illumination at the sample covered acircular region approximately 50 μm in diameter.

Microscope System

Structured illumination microscopy (SIM) imaging was performed on anOlympus IX81 inverted microscope equipped with both left and right sideports, and an automated XY stage with an additional Z piezoelectricstage (200 μm range, Applied Scientific Instrumentation, PZ-2000). Thepatterned excitation (e.g. multi-focal illumination pattern) created bythe DMD was brought in via the left side port to the microscope.Fluorescence emitted by the illuminated sample was collected by theobjective, reflected with a dichroic mirror (Chroma, zt405/488/561),passed through a 180 nm tube lens internal to the microscope, filteredappropriately to reject pump light (Semrock, LP02-488RE-25 andNF03-561E-25), and detected with a scientific-grade complementarymetal-oxide-semiconductor (sCMOS) camera (Cooke, pco.edge) mounted onthe right side port. Correctly aligning the sCMOS along the optical axiswas critical in achieving near diffraction-limited performance. To aidin the correct positioning of the camera, a 60× objectives typicallyused in imaging with a 10× air objective (Olympus, CPlanFl 10×, 0.3NA),an optic much more sensitive to errors in axial alignment. A fixedillumination pattern (similar to one used in SIM) onto the fluorescentlake sample, and translated the camera along the optical axis until theapparent size of each illumination spot was minimized.

Sample Preparation

U2OS cells were cultured on ethanol sterilized, cleaned #1.5 25 mmdiameter coverslips (Warner Instruments, 64-0715) in standard growthmedia (DMEM-HG (Invitrogen, 11960), sodium pyruvate (Invitrogen, 11360),GlutaMAX (Invitrogen, 35050) and 10% heat inactivated fetal bovine serum(Invitrogen, 11082)). To stain the samples for microtubules, cells werefixed in with a mixture of 0.5% glutaraldeyde, 0.37% formaldehyde, and0.3% Triton X-100 in Cytosketetal Buffer (CB, 10 mM MOPS, 138 mM KCl, 2mM MgCl₂, 2 mM EGTA, 0.01% NaN₃, and 160 mM Sucrose, pH 6.1). Afterfixation, the cells were washed in CB, quenched with 100 nm glycine,washed in CB, and blicked in antibody dilution buffer (AbDil, 150 mMNaCl, 20 nM Tris, 0.1% Triton X-100, 0.1% NaN₃, and 2% bovine serumalbumin, pH 7.4). The primary monoclonal antibody (Invitrogen, 32-2500)was incubated with the cells diluted to 2 μg/mL in AbDil for one hour atroom temperature. Following primary antibody incubation, the cells werewashed in the phosphor-buffered saline before incubating the cells withthe secondary, Alexa Fluor 488 labeled antibody (Invitrogen, A-11001) at1:200 dilution in AbDil for 1 hour.

Samples for dual-color experiments were initially stained withMitotracker Red (Invitrogen, M-7512) as per the manufacturer'sinstruction prior to fixation. After mitochondrial labeling, theprocedure outline above was used to stain the microtubules. All sampleswere mounted in fluoromount G (Electron Microscopy Solutionis, 17984-25)to a standard 25 mm×75 mm glass slide (SPI supplieds, #01251-AB) andsealed with nail polish.

a) Subdiffractive Beads

Yellow-green or red fluorescent beads (Invitrogen, F8803, 110 nmdiameter; Invitrogen F8801, 100 nm diameter) were used for all pointspread function (PSF) measurements. Beads were diluted from the stockconcentration of 1:1300 (1:200 in distilled water and 1:13 in ethanol)and spread over cleaned glass coverslips. After air-drying for 5 minutesto evaporate the ethanol, coverslips were washed twice in distilledwater to remove unattached beads. After air-drying again, the beads weremounted in fluoromount or silicone oil onto glass slides, and sealedwith nail polish.

b) Zebrafish Samples

Tg(XIEef1a1:dclk2-GFP)^(io008) embryos carrying the zebrafish dclk2-GFPtransgene were used in thick MSIM experiments shown in FIGS. 16A and16B. To construct, this line, plasmids containing the transgene wereinjected into one-cell zebrafish embryos along with Tol2 mRNA.Fluorescent embryos were raised to adulthood and crossed to select forgerm line transmission by screening the offspring for GFP expression.

Tg(XIEdf1a1:dclk2-GFP)^(io008) embryos were collected by naturalspawning and maintained at 28 degrees Centigrade. Prior to imaging bythe multi-focal SIM system 200, embryos at 24 hpf were anesthetized inTricaine (Sigma, E105210 at a final concentration of 600 μM in embryomedia (60 mg Instant ocean sea salt (PetSmart) per liter ddH₂O).Anesthetized embryos were mounted on round coverslips, immobilized in 1%low-melt agarose (Cambrex, 50080), placed in a round coverslip holder(ASI, I-3033-25D), covered with embryo media, and imaged at roomtemperature.

Data Processing

Following acquisition of raw images using the illumination andmicroscope system described above, each set of collected raw images ofthe samples were processed into a super-resolution image using theprocessing system 225 having software written in the Python programminglanguage. The processing steps employed by the processing system 225were: (i) Automatic lattice detection to precisely determine therespective locations of the illumination spots; (ii) Digital pinholemasking around each detected illumination spot to reject out-of-focuslight, and optical flat-fielding using calibration data; (iii) Localcontraction (e.g., scaling), and re-sampling the area around eachillumination spot to improve the resolution by √2; (iv) Summing theprocessed raw images to produce a super-resolution composite image; and(v) Using conventional deconvolution techniques to recover the full 2×resolution enhancement. These process steps are discussed in greaterdetail above with respect to the focusing, scaling, and summingoperation 210 executed by the processing system 225 of the multi-focalSIM system 200.

It should be understood from the foregoing that, while particularembodiments have been illustrated and described, various modificationscan be made thereto without departing from the spirit and scope of theinvention as will be apparent to those skilled in the art. Such changesand modifications are within the scope and teachings of this inventionas defined in the claims appended hereto.

What is claimed is:
 1. A microscopy system comprising: a light sourcefor transmitting a single light beam; a first spinning disk withconverging microlens for rotation in a first direction and positionedalong an optic axis for splitting the single light beam into a pluralityof light beams forming a multi-focal pattern; a second spinning diskwith a pinhole array for rotation in the first direction and positionedalong the optic axis for blocking out-of-focus light beams of theplurality of light beams for each multi-focal pattern and allowingthrough in-focus light beams of the plurality of light beams to passthrough the spinning disk with a pinhole array, and a third spinningdisk with a diverging microlens for rotation in the first direction andpositioned along the optic axis, wherein the first spinning disk withconverging microlens, the second spinning disk with a pinhole array, andthe third spinning disk with a diverging microlens rotate in syncrelative to each other.
 2. The microscopy system of claim 1, wherein thethird spinning disk with diverging microlens is half the focal length ofthe first spinning disk with converging microlens.
 3. The microscopysystem of claim 1, wherein third spinning disk with diverging microlenshas the same number of microlenses as the first spinning disk withconverging microlens.
 4. The microscopy system of claim 1, wherein themicrolenses of the third spinning disk with a diverging microlens arespaced at the same spatial location as the first spinning disk with aconverging microlens.
 5. The microscopy system of claim 1, wherein thefirst spinning disk with a converging microlens, the second spinningdisk with a pinhole array, and the spinning disk with a divergingmicrolens are spaced apart from each other by the difference of focallengths of the first spinning disk with a converging microlens, thesecond spinning disk with a pinhole array, and the spinning disk with adiverging microlens such that a Galilean telescope with a magnificationof one half is arranged.